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Abstract 

The  research  presented  here  focuses  on  applying  the  RF  “Distinct  Native  Attribute” 
(RF-DNA)  fingerprinting  process  to  Programmable  Fogic  Controller  (PFC)  hardware 
devices  as  a  means  of  device  discrimination  to  mitigate  risk  of  an  attack  on  Supervisory 
Control  And  Data  Acquisition  (SCADA)  systems.  A  previously  developed  signal  collection 
method  was  implemented  to  collect  Unintentional  Radiated  Emission  (URE)  from  ten  Allen 
Bradley  SFC-500  PECs  using  a  National  Instruments  collection  platform  for  comparison  of 
results  against  collections  taken  using  a  Lecroy  collection  platform.  RF-DNA  fingerprints 
were  generated  using  Time-Domain  (TD)  features  and  used  for  device  classification  (a  one- 
to-many  looks  “most  like”  comparison)  and  verification  (a  one-to-one  looks  “how  much 
like”  comparison). 

Results  are  presented  for  two  classification  processes,  the  Generalized  Relevance 
Learning  from  Vectors  Quantized  Improved  (GRLVQI)  and  Multiple  Discriminant 
Analysis  Maximum  Likelihood  (MDA/ML)  processes.  GRLVQI  feature  relevance 
rankings  are  used  here  for  Quantitative  feature  Dimensional  Reduction  Analysis  (DRA), 
i.e.  removing  all  but  the  most  influential  features  while  still  achieving  the  desired 
classification  and  verification  performance.  Qualitative  feature  DRA  is  also  used  by 
constructing  feature  sets  solely  comprised  of  one  TD  signal  response  attribute,  i.e. 
amplitude,  phase,  frequency. 

Using  the  Lecroy  collection  platform  the  full  dimensional  feature  set  demonstrated  aver¬ 
age  classification  accuracy  of  Cave  >90%  for  the  1)  MDA/ML  classifier  at  S  NR  >6.5  dB  2) 
GRLVQI  classifier  at  SNR  >11  dB.  The  National  Instruments  collection  platform  demon¬ 
strated  average  classification  accuracy  of  Cave  >90%  for  the  1)  MDA/ML  classifier  at 
SNR  >16.5  dB  2)  GRLVQI  classifier  at  SNR  >17  dB.  This  corresponds  to  a  Lecroy  gain 
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of  G\sot?=10  dB  in  the  MDA/ML  classifier  and  a  gain  of  GSNR= 6  dB  in  the  GRLVQI  clas¬ 
sifier. 

For  the  Lecroy  platform  data  at  SNR=10  dB,  using  the  MDA/ML  classifier  satisfied  the 
EER  >10%  benchmark  for  100%  of  PLC  devices  for  both  Authorized  Device  Identification 
and  Rogue  Device  Rejection.  The  Lecroy  platform  data,  at  SNR=10  dB,  using  the  GRLVQI 
classifier,  3  of  5  devices  satisfied  the  EER  >10%  benchmark  for  Authorized  Device 
Identification  and  100%  of  devices  for  Rogue  Device  Rejection  satisfied  the  EER  >10% 
benchmark.  The  National  Instruments  data,  at  SNR=20  dB,  using  the  MDA/ML  classifier, 
3  of  5  devices  satisfied  the  EER  >10%  benchmark  for  Authorized  Device  Identification 
and  100%  of  devices  for  Rogue  Device  Rejection  satisfied  the  EER  >10%  benchmark.  The 
National  Instruments  data,  at  SNR=20  dB,  using  theGRLVQI  classifier,  2  of  5  devices  for 
Authorized  Device  Identification  and  100%  of  devices  for  Rogue  Device  Rejection  satisfied 
the  EER  >10%  benchmark.  The  similar  results  between  collection  platforms  represent  a 
Gsnr  *10  dB  gain  using  the  Lecroy  receiver  over  the  National  Instruments  receiver. 


v 


Table  of  Contents 


Page 

Abstract .  iv 

Table  of  Contents .  vi 

List  of  Figures .  ix 

List  of  Tables .  xi 

List  of  Acronyms . xii 

I.  Introduction .  1 

1.1  Research  Motivation  .  1 

1.2  Research  Approach .  4 

1.2.1  Emission  Collection .  4 

1.2.2  Fingerprint  Generation .  4 

1.2.3  Device  Classification .  5 

1.2.4  Device  ID  Verification .  5 

1.2.5  Cross  Platform  Validation .  6 

1.3  Research  Contributions .  6 

1.4  Document  Organization .  6 

II.  Related  Work/Literature  Review .  8 

2.1  SCADA  Systems .  8 

2.1.1  Programmable  Logic  Controllers .  8 

2.1.2  Ladder  Logic  Programs .  9 

2.1.3  Vulnerabilities  .  9 

2.2  RF  Signal  Collection .  11 

2.2.1  Radiated  Emissions .  11 

2.2.2  Correlation .  12 

2.3  Device  Discrimination  .  12 

2.3.1  Classification .  12 

2.3. 1.1  MDA/ML .  13 

2.3. 1.2  GRLVQI .  13 

2.3.2  Verification .  14 


vi 


Page 

III.  Methodology  .  16 

3.1  PLC  Device  Description  .  16 

3.2  RF  Signal  Collection .  18 

3.2.1  PLC  Collection  Configuration .  18 

3.2.2  RF  Probe  Placement .  19 

3.2.3  Sampling  and  Triggering .  20 

3.3  Post  Collection  Processing .  21 

3.4  ROI  Extraction .  23 

3.5  Fingerprint  Generation .  24 

3.6  Feature  Set  Dimensional  Reduction  .  26 

3.6.1  Qualitative .  27 

3.6.2  Quantitative .  27 

3.7  Device  Discrimination  .  28 

3.7.1  Classification .  28 

3.7.2  Verification .  29 

3.8  Performance  Evaluation .  29 

IV.  Results .  31 

4.1  Expansion  of  Lecroy  Platform  RF-DNA  Fingerprinting  Results .  32 

4.1.1  Full  Dimensional .  32 

4.1.2  Dimensional  Reduction .  32 

4.2  National  Instruments  Platform  RF-DNA  Fingerprinting  Results  .  35 

4.2.1  Full  Dimensional .  35 

4.2.2  Dimensional  Reduction .  36 

4.3  Device  Verification .  39 

4.3.1  Authorized  Device  Identification .  39 

4.3.2  Rogue  Device  Identification .  41 

4.4  Cross  Receiver  Validation .  42 

V.  Conclusion  .  46 

5.1  Research  Summary .  46 

5.1.1  Cross-Platform  Validation .  48 

5.1.2  Dimensional  Reduction  Analysis .  48 

5.2  Future  Work  Recommendations  .  49 

VI.  Appendix .  51 


vii 


Bibliography 


Page 
.  61 


List  of  Figures 


Figure  Page 

1.1  OSI  Model  .  3 

2.1  Ladder  Logic  Program .  10 

2.2  MDA/ML  Class  Boundaries  .  14 

2.3  MDA/ML  Projection  Matrices .  14 

2.4  GRLVQI  Feature  Space .  15 

2.5  GRLVQI  Relevance  Rankings  .  15 

3.1  PLC  Radiated  Emission  Spectral  Intensities .  18 

3.2  Bandpass  Filter  Magnitude  Response .  22 

3.3  Region  Of  Interest  Extraction .  24 

3.4  RF-DNA  Fingerprint  Sequence  Generation  .  25 

4.1  Lecroy:  Full  Dimensional  Classification  Results .  33 

4.2  Lecroy  Platform  DRA  Testing  Results .  34 

4.3  Relevance  Rankings .  36 

4.4  National  Instruments:  Full  Dimensional  Classification  Results .  37 

4.5  National  Instruments  Platform  DRA  Testing  Results .  38 

4.6  MDAML  Authorized  ID  Verification  Results  .  40 

4.7  GRLVQI  Authorized  ID  Verification  Results .  40 

4.8  MDAML  Authorized  ID  Verification  Results  .  41 

4.9  GRLVQI  Authorized  ID  Verification  Results .  42 

4.10  MDA/ML  Authorized  ID  Verification  Results .  43 

4.11  GRLVQI  Authorized  ID  Verification  Results .  43 

4.12  MDA/ML  Authorized  ID  Verification  Results .  44 

4.13  GRLVQI  Authorized  ID  Verification  Results .  44 


IX 


Figure  Page 

4.14  National  Instruments  Authorized  ID  results  at  the  collected  SNR . 45 

4.15  National  Instruments  Rogue  Rejection  results  at  the  collected  SNR . 45 

6.1  Lecroy:  Qualitative  (Amplitude)  Classification  Results .  51 

6.2  Lecroy:  Qualitative  (Phase)  Classification  Results .  52 

6.3  Lecroy:  Qualitative  (Frequency)  Classification  Results  .  53 

6.4  Lecroy:  Quantitative  (Top  33%)  Classification  Results  .  54 

6.5  Lecroy:  Quantitative  (Top  10%)  Classification  Results  .  55 

6.6  National  Instruments:  Qualitative  (Amplitude)  Classification  Results .  56 

6.7  National  Instruments:  Qualitative  (Phase)  Classification  Results  .  57 

6.8  National  Instruments:  Qualitative  (Frequency)  Classification  Results .  58 

6.9  National  Instruments:  Quantitative  (Top  33%)  Classification  Results .  59 

6.10  National  Instruments:  Quantitative  (Top  10%)  Classification  Results .  60 


x 


List  of  Tables 


Table  Page 

1.1  Research  Contributions .  7 

3.1  Receiver  Collection  Platforms  .  16 

3.2  PLC  Hardware  Device  Labellings  .  17 

3.3  Feature  Set  Dimensionality .  27 


List  of  Acronyms 


Acronym  Definition 

AFRL  Air  Force  Research  Laboratory 
AFIT  Air  Force  Institute  of  Technology 
AWGN  Additive  White  Gaussian  Noise 

COTS  Commercial  Off  The  Shelf 

DHT  Discrete  Hilbert  Transform 

DRA  Dimensional  Reduction  Analysis 

DUT  Device  Under  Test 

EER  Equal  Error  Rate 

FPGA  Field  Programmable  Gate  Array 

FD  Frequency  Domain 

FVR  False  Verification  Rate 

GRLVQI  Generalized  Relevance  Learning  from  Vectors  Quantized  Improved 
HT  Hilbert  Transform 
IC  Integrated  Circuits 
ICS  Industrial  Control  Systems 


IF  Intermediate  Frequency 


Acronym  Definition 

IRE  Intentional  Radiated  Emission 

LLP  Ladder  Logic  Program 

LPL  Low  Pass  Lilter 

MCU  Microcontroller  Unit 

MDA/ML  Multiple  Discriminant  Analysis  Maximum  Likelihood 
NI  National  Instruments 

NISAC  National  Infrastructure  Simulation  and  Analysis  Center 

ORNL  Oak  Ridge  National  Laboratory 

OSI  Open  Systems  Interconnection 

PLC  Programmable  Logic  Controller 

PML  Probability  Mass  Lunction 

RL  Radio  Lrequency 

RL-DNA  Radio  Lrequency  Distinct  Native  Attribute 
RAR  Rogue  Accept  Rate 
ROC  Receiver  Operating  Characteristic 
ROI  Region  Of  Interest 

SCADA  Supervisory  Control  And  Data  Acquisition 


SNR  Signal  to  Noise  Ratio 


Acronym  Definition 
TVR  True  Verification  Rate 

TD  Time-Domain 

URE  Unintentional  Radiated  Emission 
WPAFB  Wright  Patterson  Air  Force  Base 


xiv 


PLC  HARDWARE  DISCRIMINATION  USING  RF-DNA  FINGERPRINTING 


I.  Introduction 

This  chapter  introduces  the  research  topic  and  describes  the  approach  taken  to  attain  the 
research  goals.  Section  1 . 1  gives  an  overview  of  Supervisory  Control  And  Data  Acquisition 
(SCADA)  systems  and  some  of  the  issues  and  vulnerabilities  pertaining  to  them.  Section 
1.2  describes  the  approach  taken  to  implement  the  AFIT  Radio  Frequency  -  Distinct 
Native  Attribute  (RF-DNA)  process  relative  to  semi-conductor  devices  and  unintentional 
emissions.  Section  1.3  provides  a  reference  for  current  and  related  research  efforts. 

1.1  Research  Motivation 

Today  electronic  systems  are  present  in  everyday  life.  It  would  be  nearly  impossible  to 
go  outside  in  any  urban  environment  or  any  modern  day  office  environment  and  not  witness 
an  electronic  system  of  some  kind.  With  the  proliferation  of  Information  Technology 
(IT)  systems,  large  networks  such  as  the  internet,  cellular  phone  networks,  and  modern 
television  are  seemingly  commonplace.  Fess  publicly  discussed  are  the  IT  networks  used 
to  operate  national  critical  infrastructure  such  as  the  networks  used  in  nuclear  power 
generation  plants,  waste  water  treatment,  traffic  grids,  and  sewage  systems.  These  networks 
are  also  commonplace  and  have  been  identified  as  a  cybersecurity  vulnerability  [53]. 

A  type  of  system  often  used  to  control  operations  of  national  critical  infrastructure  is 
a  SCADA  system.  SCADA  systems  are  essentially  miniature  computer  systems  used  to 
control  industrial  processes.  A  Programmable  Fogic  Controller  (PFC)  is  the  most  basic 
unit  of  a  SCADA  system  and  is  used  for  controlling  a  particular  automated  process  such  as 
temperature  or  pressure  monitoring.  . 
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One  of  the  main  types  of  physical  components  in  PLCs,  as  with  virtually  all  electronic 
devices,  are  Integrated  Circuits  (IQs.  IC  devices  such  as,  Field  Programmable  Gate 
Array  (FPGA)s,  operational  amplifiers,  and  microcontrollers  are  widely  used  and  often 
manufactured  overseas  as  a  method  of  cost  reduction.  The  majority  of  ICs  used  in  modern 
military  systems  are  made  off-shore  [10].  ICs  can  be  counterfeited,  or  embedded  with 
hardware  trojans  [1,  10]. 

Industrial  Control  Systems  (ICS)  can  fall  prey  to  such  IC  hardware  vulnerabilities. 
A  counterfeited  device  or  a  device  that  has  been  unknowingly  altered,  that  is  used  in 
control  systems  for  critical  applications  poses  a  significant  vulnerability.  Furthermore, 
there  is  in  increasing  reliance  upon  ICS  networks  and  particularly  SCADA  systems 
to  control  and  monitor  critical  process  [40].  Although  critical  infrastructure  may  be 
owned  by  private  companies  or  corporations,  government  also  has  a  reliance  on  national 
critical  infrastructure.  This  co-dependence  led  to  the  formation  of  National  Infrastructure 
Simulation  and  Analysis  Center  (NISAC),  a  program  within  the  Department  of  Homeland 
Security  (DHS)  whose  mission  is  to  research  and  analyze,  through  modelling  and 
simulation,  vulnerabilities  and  complexities  of  critical  infrastructure  [5,  41]. 

Security  measures  such  as  bit  level  credentials  used  for  digital  device  authentication 
including  Media  Access  Control  (MAC)  addresses  and  International  Mobile  Equipment 
Identity  (IMEI)  numbers  exist  as  measures  of  security.  When  considering  the  Open 
Systems  Interconnection  (OSI)  model,  these  measure  of  security  are  at  the  implemented 
at  Application  (Layer  1)  or  Network  (Layer  5)  layers.  These  are  far  from  infallible  and 
there  exist  methods  of  bypassing  these  layers  of  security  [33,  51].  PLC  Operating  Systems 
often  use  proprietary  communication  protocols  and  are  connected  in  vast  networks.  PLCs 
themselves  have  limited  processing  power  and  memory  availability.  Because  of  the  nature 
of  their  implementation  and  operating  characteristics,  they  are  often  limited  in  regards  to 
defensive  monitoring  software  such  as  anti-intrusion  and  anti-virus  software. 
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Figure  1.1:  OSI  7  Layers  network  model  [48] 


Furthermore  it  is  not  uncommon  for  a  SCADA  system  by  remain  in  service  for  decades. 
For  this  reason  they  become  obsolete  to  modern  security  standards  and  capabilities.  PLC 
devices  remain  vulnerable  to  hardware  trojans,  substitutes  and  counterfeits. 

Although  work  has  been  done  at  securing  PLC  devices  at  the  higher  layers  of  the  OSI 
model,  comparatively  little  work  has  been  done  at  the  lowest  layer  i.e.  the  physical 
waveform  layer.  This  research  augments  hardware  device  security,  in  particular  PLC  IC 
devices,  by  means  of  verifying  authenticity  at  the  physical  layer.  While  PLCs  are  used  as  a 
proof  of  concept  for  hardware  devices  discrimination,  the  topics  contained  herein  apply  to 
the  majority  of  semi-conductor  based  devices. 

1.2  Research  Approach 

The  goal  of  this  research  is  to  use  Unintentional  Radiated  Emission  (URE)  produced  by 
IC  devices  as  a  means  to  discriminate  between  PLC  devices.  Inside  a  physical  PLC  device 
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there  are  many  points  where  URE  may  be  collected.  Collected  emissions  are  taken  from 
the  microcontroller  within  the  PLC.  Previous  research  efforts  have  shown  this  region  to  be 
viable  for  collecting  device  URE  [43].  The  collected  emissions  are  used  to  develop  Radio 
Frequency  Distinct  Native  Attribute  (RF-DNA)  fingerprints.  The  fingerprints  are  used  to 
distinguish  devices  by  exploiting  Radio  Frequency  (RF)  emission  characters  unique  to  a 
device  that  are  caused  by  its  component  manufacturing  variations. 

Another  goal  is  to  reliably  reduce  the  dimensionality  of  RF-DNA  fingerprint  data  sets. 
Dimensional  reduction  allows  for  faster  execution  time  and  may  mitigate  adverse  affects 
on  classification  performance  caused  by  noisy,  irrelevant  or  redundant  information  [3,  24]. 
It  is  expected  that  dimensional  reduction  will  reduce  execution  time  with  the  potential  to 
improve  classification  performance. 

1.2.1  Emission  Collection 

Using  RF  signal  characteristics  as  means  of  device  authentication  as  been  widely 
researched  [2,  4,  7,  9,  12,  13,  15-17,  19,  21,  36,  38,  43,  46,  49].  Although  research  has 
been  done  using  both  Intentional  Radiated  Emission  (IRE)  and  URE,  URE  has  not  been  as 
well  researched.  The  URE  signals  used  for  device  discrimination  differ  from  IRE  signals 
in  that  they  are  not  intentionally  broadcast  and  therefore  have  much  lower  average  signal 
power  and  do  not  adhere  to  a  specified  broadcast  pattern.  IRE  and  URE  have  collection 
specific  configurations  accounting  for  required  bandwidth  and  center  frequency  which  is 
largely  determined  by  the  Device  Under  Test  (DUT). 

1.2.2  Fingerprint  Generation 

Collected  signals  first  undergo  post-collection  digital  processing  and  are  then  used  to 
develop  fingerprints  using  Air  Force  Institute  of  Technology  (AFIT)’s  RF-DNA  process 
[4,  9,  36,  43].  The  fingerprints  are  constructed  from  statistical  attributes  of  the  Time- 
Domain  (TD)  signal  responses:  amplitude,  phase  and  frequency.  The  statistics  used 
are:  standard  deviation  (cr),  variance  (cr2),  skewness  (y),  and  kurtosis  (a:).  Other  signal 
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features  have  been  used  in  previous  AFIT  research  such  as  Frequency  Domain  and  Gabor 
Transform,  however  this  research  only  considers  TD  signal  responses  of  URE  signal 
collections. 

1.2.3  Device  Classification 

In  classification  a  process  referred  to  as  a  classifier  uses  RF-DNA  fingerprints  from 
known  devices  to  train  or  develop  a  classification  model.  This  model  represents  the  known 
devices  (Authorized  Devices)  fingerprint  characteristics.  Using  the  model,  unknown  device 
fingerprints  are  classified  or  aligned  (correctly  or  incorrectly)  to  a  particular  known  device 
represented  in  the  model. 

Device  classification  allows  a  one-to-many  device  comparison.  Devices  that  are  not 
represented  in  the  model  (Rogue  Devices)  will  still  be  classified  as  one  of  the  Authorized 
Devices  i.e.  all  devices  will  be  classified  as  one  of  the  known  Authorized  devices. 
Therefore  a  verification  method  is  used  to  evaluate  ”how  much  like”  a  device  resembles 
a  selected  class. 

1.2.4  Device  ID  Verification 

Verification  is  a  one-to-one  comparison  of  fingerprints  for  an  unknown  device  to 
fingerprints  of  a  known  Authorized  device.  The  verification  process  is  implemented  for 
two  scenarios:  Authorized  Device  Identification  and  Rogue  Device  Rejection. 

Authorized  Device  Identification  examines  how  much  like  an  Authorized  Device  looks 
like  a  different  Authorized  Device.  Rogue  Device  Rejection  is  a  comparison  of  how  much 
like  a  rogue  device  resembles  an  Authorized  Device.  The  intent  is  for  the  model  to  be  able 
to  clearly  distinguish  the  Authorized  Devices  from  each  other,  and  correctly  discriminate 
between  Rogue  and  Authorized  devices.  Previous  researchers  have  been  able  to  use  the 
general  verification  process  using  RF-DNA  fingerprints  to  verify  PLC  microcontroller 
devices  with  better  than  99.5%  accuracy  [7]. 
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1.2.5  Cross  Platform  Validation 

To  validate  the  repeatability  of  the  signal  collection  and  fingerprinting  process,  two 
different  collection  platforms  are  used  for  signal  collection.  The  two  collection  platforms 
are  detailed  in  Table  3.1.  The  same  collection  method  was  used  for  both  receiver  platforms, 
as  well  the  devices  collected  against,  and  all  supporting  equipment.  The  results  from  the 
two  collection  platforms  are  shown  in  Chapter  4.  They  are  first  presented  independently, 
and  are  then  compared  directly. 

1.3  Research  Contributions 

The  research  goal  includes  expanding  upon  previous  AFIT  fingerprinting  results, 
and  also  implementing  and  verifying  the  signal  collection  method  in  [43]  by  replicating 
the  process  with  another  receiver.  Previous  AFIT  results  were  expanded  by  examining 
the  effects  of  feature  dimensional  reduction  for  both  classification  and  verification ,  as 
well  as  the  addition  of  another  classifer,  the  Multiple  Discriminant  Analysis  Maximum 
Likelihood  (MDA/ML)  classifier.  Summarized  below  are  the  research  contributions  and 
findings  related  to  PLC  device  hardware  discrimination. 

1.4  Document  Organization 

The  remainder  of  the  document  is  organized  as  follows.  Chapter  2  discusses 
SCADA  system  vulnerabilities,  Ladder  Logic,  Correlation  based  processing,  and  the 
classification/verification  process  using  the  MDA/ML  and  GRLVQI  classifiers.  Chapter 
3  details  the  implemented  signal  collection  process,  post-collection  processing,  and 
fingerprint  generation  as  well  as  feature  dimensional  reduction.  Chapter  4  shows  the 
results  of  PLC  hardware  discrimination  using  RF-DNA  fingerprinting.  Chapter  5  provides 
a  summary  of  the  findings  as  well  as  potential  future  work. 
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Table  1.1:  Relational  mapping  between  technical  areas  of  “Previous  Work”  and  AFIT 
research,  and  “Current  Research”  contributions. 


Technical  Area  Previous  Work  Current  Research 


Addressed 

Ref# 

Addressed  # 

TD  Features 

X 

[31,32,  36,  37] 
[46,  47,  49,  50] 

X 

SD  Features 

X 

[7,  9,  39.  49] 

CD  Features 

X 

[46,  47] 

Emission  Type 


Intentional  (IRE) 

X 

[31,32,  36,  37] 
[46,  47,  49,  50] 
[17,25,27,28] 

Unintentional  (URE) 

X 

[6,  7,  9,  43,  44] 

X 

Burst 

X 

[31,32,  36,  37] 
[46,  47,  49,  50] 
[17,25,27,28] 

Continuous 

X 

[6,  7,  9,  43,  44] 

X 

High  SNR 

X 

[31,32,  36,  37] 
[46,  47,  49,  50] 
[17,25,27,28] 

Eow  SNR 

X 

[6,  7,  9,  43,  44] 

X 

Classification/Verification  Processes 


MDA/ME 

X 

[31,32,  36,  37] 
[46,  47,  49,  50] 
[17,25,27,28] 

X 

GREVQI 

X 

[31,32,  36,  37] 

X 

EFS 

X 

[25-28] 

Dimensional  Reduction  Analysis  (DRA) 


MDA/ME 

X 

[31,32,  36,  37] 

X 

GRLVQI 

X 

[30,  36,  37] 

X 

LFS 

X 

[25-28] 

Verification 


Electronic  Components 

X 

[6,  7,  9,  43,  44] 

X 

Authorized  Wireless  Devices 

X 

[17,  36,  37] 

Rogue  Wireless  Devices 

X 

[17,  36,  37] 

Device  Operations 

X 

[43-45] 
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II.  Related  Work/Literature  Review 


This  chapter  gives  background  information  on  Programmable  Logic  Controller  (PLC)s 
and  device  fingerprinting  and  discusses  supporting  research  and  associated  academic 
works.  Section  2.1  details  the  significance  of  PLCs  in  the  context  of  Supervisory  Control 
And  Data  Acquisition  (SCADA)  systems  as  well  as  PLC  and  SCADA  vulnerabilities. 
Section  2.2  gives  a  description  of  the  approach  and  challenges  of  signal  collection  for 
PLCs  and  details  the  utilization  of  multiple  collection  platforms.  Section  2.3  describes 
PLC  device  classification  and  verification. 

2.1  SCADA  Systems 

SCADA  systems  are  used  to  automate  and  control  large  scale  industrial  applications  such 
as:  power  generation  plants,  traffic  grids,  and  waste  water  removal  systems.  They  consist 
of  a  multitude  of  devices  including  PLCs  and  Remote  Terminal  Units  (RTUs).  Originally 
SCADA  systems  used  dedicated  wires  for  communication  between  devices.  Although 
wired  communications  are  still  used  today,  wireless  SCADA  systems  have  become  widely 
used,  particularly  in  remote  sensing  and  control  environments. 

The  earliest  SCADA  systems  used  in  the  1960s  were  first  used  in  power  generation 
plants  to  monitor  and  control  sub-stations.  Over  the  last  50  years  SCADA  systems 
have  significantly  evolved  as  computer  processing  power  and  component  size  continue 
to  progress.  However  SCADA  systems  can  have  a  lifetime  on  the  order  of  decades  and 
many  legacy  systems  often  do  not  have  the  processing  capabilities  and  to  run  modern  day 
anti-intrusion  detection  systems  [14]. 

2.1.1  Programmable  Logic  Controllers 

A  particular  component  of  a  SCADA  system  that  is  used  to  collect  sensor  data  and 
control  electro-mechanical  operations  is  a  PLC,  the  device  to  which  this  research  is  focused. 
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PLCs  are  used  to  perform  low-level  operations  within  a  SCADA  system,  such  as  sensory 
data  input  and  output,  and  were  originally  designed  to  replace  physical  relays.  Individual 
PLC  devices  are  often  referred  to  as  modules.  Modules  can  be  specialized  for  certain 
applications  e.g.  power,  I/O,  as  well  as  specific  types  of  sensor  modules.  Some  large 
SCADA  systems  (e.g.  a  power  grid)  can  be  comprised  of  hundreds  if  not  thousands  of 
PLCs  and  supporting  units  [22]. 

2.1.2  Ladder  Logic  Programs 

PLCs  perform  required  process  using  a  program  called  a  Ladder  Logic  Program 
(LLP).  The  name  Ladder  Logic  originally  refereed  to  relay  logic  schematics  used  in  control 
and  manufacturing  [35] .  In  the  advent  of  the  digital  age  Ladder  Logic  now  commonly  refers 
to  the  widely  used  programming  language  used  for  programming  PLCs.  LLPs  are  executed 
by  a  PLC  in  what  is  called  a  Ladder  Logic  scan.  At  the  beginning  of  a  scan  the  PLC  first 
reads  all  input  values.  It  then  performs  the  operations  on  the  top-most  “rung”,  sequentially 
executing  all  rungs.  It  then  assigns  all  output  values.  An  example  LLP  is  depicted  in  Fig. 
2.1. 

In  real  world  applications  LLPs  can  be  recursive  and  complex,  containing  loops  and 
jumps.  Consider  a  traffic  light  program  continually  looping  through  traffic  fight  patterns, 
or  the  complexity  of  a  power  generation  plant.  However  the  LLPs  used  in  this  research 
are  intentionally  non-recursive,  i.e.  there  are  no  internal  loops.  The  programs  themselves 
are  very  basic  consisting  of  N0p  <10  operations.  This  is  done  purposefully  to  ensure  an 
experimentally  repeatable  signal  collection  process  across  multiple  PLC  devices. 

2.1.3  Vulnerabilities 

As  previously  mentioned,  PLCs  can  have  an  operational  lifetime  of  several  decades.  Due 
to  their  age  many  PLCs  do  not  have  the  computer  processing  capability  required  to  run 
modern  intrusion  prevention  and  security  software.  This  leaves  many  PLCs  vulnerable  to 
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End  of  Scan 


Figure  2.1:  LLP  example  program  showing  a  single  MOV  and  SQR  operation  [43]  preceded 
by  two  logic  rungs. 


cyber  attack.  A  well  known  example  of  such  an  attack  is  Stuxnet,  which  exploited  security 
vulnerabilities  and  injected  malicious  code  into  SCADA  systems  [53]. 

Extensive  research  has  been  done  attempting  to  secure  SCADA  systems.  Existing 
security  measures  use  bit  level-credentials  such  at  the  Media  Access  Control  (MAC) 
address  and  the  International  Mobile  Equipment  Identity  (IMEI)  numbers  to  control  access 
to  a  network  while  other  software  systems  are  used  to  protect  against  malware.  However 
many  of  these  measures  and  methods  are  not  implemented  in  current  SCADA  systems 
and,  in  particular,  PLCs.  Even  if  implemented  SCADA  hardware  may  still  be  vulnerable 
to  hardware  trojans  and  counterfeits.  An  alternative  to  bit-level  credentials  has  emerged 
using  Radio  Frequency  (RF)  radiated  emissions,  (unintentional  or  intentional)  to  extract 
unique  characteristic  device  information  at  the  physical  waveform  level  that  can  be  used  to 
discriminate  between  hardware  devices.  This  method  has  been  shown  to  be  succesful  in  a 
large  scope  of  research,  [4,  7,  9,  11-13,  15-20,  23,  25,  27,  38,  39,  43,  46,  49]. 
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2.2  RF  Signal  Collection 
2.2.1  Radiated  Emissions 

RF-DNA  fingerprints  used  to  discriminate  among  devices  are  constructed  from  captured 
radiated  emissions  from  a  given  Device  Under  Test  (DUT).  Previous  research  can  be 
categorized  into  two  types  of  radiated  emissions:  Intentional  Radiated  Emission  (IRE)  and 
Unintentional  Radiated  Emission  (URE).  IRE  RF  energy  is  intentionally  broadcasted  and 
is  engineered  to  carry  information.  Typically  IRE  RF  communication  signals  have  well 
defined  regions  such  as  a  preamble  or  payload. 

URE  RF  energy  is  leaked  electromagnetic  energy  produced  during  DUT  operation. 
UREs  by  comparison  are  not  engineered  or  well  structured.  This  creates  an  added 
challenge  of  repeatability  when  collecting  URE.  Radio  Frequency  Distinct  Native  Attribute 
(RF-DNA)  IC  device  fingerprinting  exploits  characteristic  differences  in  device  waveforms 
caused  by  variances  in  manufactured  devices.  These  characteristic  differences  are  identified 
by  calculating  statistics  for  device’s  waveform  attributes  with  the  assumption  that  the 
waveforms  being  fingerprinted  have  the  same  structure.  The  IC  components  collected 
against  are  often  shielded  to  mitigate  RF  interference  to  and  from  other  components. 
Capturing  repeated  waveforms  from  such  components  requires  added  measures  compared 
when  to  IRE  collections. 

URE  signal  collections  share  physical  aspects  with  as  IRE  collections  however  URE 
collections  often  require  different  equipment  and  collection  procedures.  Collections  can  be 
invasive,  or  non-invasive.  One  example  of  a  non-invasive  technique  is  using  an  Electro 
Magnetic  (EM)  probe.  Previous  research  efforts  have  used  non-invasive  techniques  to 
capture  IC  electrical  responses  directly  from  connecting  pins.  Information  exploited  from 
this  method  includes  (power,  timing,  control,  data,  etc.).  Although  these  methods  are  non- 
invasive  they  do  require  contact,  i.e.  a  physical  connection  is  required  [2,  21].  Whereas 
RF-based  methods  utilize  an  EM  probe  in  close  proximity  to  the  DUT  [29] . 
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2.2.2  Correlation 


As  mentioned,  this  research  makes  use  of  URE  which  is  not  well  structured  or 
engineered.  A  method  to  identify  an  Region  Of  Interest  (ROI)  is  necessary  (where  an  ROI  is 
analogous  to  a  communication  burst).  A  correlation  based  extraction  process  developed  in 
[43]  is  implemented  for  ROI  determination  and  extraction.  The  extraction  process  is  based 
on  a  matched  filter  implementation  which  often  used  for  the  estimation  of  communication 
symbols  in  digital  communication  systems  [42].  The  autocorrelation  (/?„[£])  and  cross¬ 
correlation  (Rxy[k])  operations  discussed  in  Section  3.3  are  defined  here, 

R*Ak\  =  YjXn<-k  (2.1) 

n 

Rx}m  =  YjXnfn-k  (2-2) 

n 

Cross  correlation  is  used  in  most  modern  day  wireless  communication  systems  as  a 
means  of  signal  detection.  Here  it  is  used  as  ROI  detection.  Similar  to  how  a  matched  filter 
is  implemented. 

2.3  Device  Discrimination 

2.3.1  Classification 

Using  RF-DNA  fingerprints  classification  is  the  process  by  which  a  given  DUT  is 
identified.  The  fingerprints  from  known  devices  are  used  in  the  classification  process  to 
develop,  or  train,  a  classification  model.  The  established  model  is  then  used  to  align  a 
DUT  fingerprint  to  one  of  the  known  devices  characterized  in  the  classification  model.  One 
of  the  research  goals  is  to  use  classification  to  correctly  identify  PLC  hardware  devices. 
Two  model  development  processes  or  classifiers  are  considered  and  are  briefly  discussed 
in  Sections  2. 3. 1.1-2. 3. 1.2. 
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2.3.1. 1  MDA/ML 

An  overview  of  the  Multiple  Discriminant  Analysis  Maximum  Likelihood  (MDA/ML) 
process  is  given  here,  and  is  implemented  as  described  in  [9].  The  MDA/ML  process  is  an 
extension  of  Fisher’s  two  class  linear  discriminant  analysis  to  Nc- 1  classes.  The  research 
presented  here  considers  Nc=5  classes  (5  PLC  hardware  devices,  Authorized  Devices).  The 
MDA/ML  classifier  projects  vectors  defined  by  individual  device  fingerprints  F  using  the 
projection  matrix  W.  Where  W  is  the  optimal  projection  matrix  that  maximizes  inter-class 
distance,  and  minimizes  intra-class  distance. 

Ff  =  WrF  (2.3) 

A  device  is  aligned  to  one  of  the  Nc  classes  based  upon  maximum  likelihood  conditional 
posterior  probability  with  the  assumption  of  equal  probabilities,  where  likelihood  is 
estimated  for  each  device’s  projected  fingerprints’  assuming  a  multivariate  Gaussian 
distribution  [9].  Figure  2.2  shows  a  visual  representation  for  a  Nc- 1=2  dimensional  feature 
space,  and  Figure  2.3  shows  the  class  projections  resulting  from  projection  matrices  Wj 
and  W2  respectively.  In  this  example,  projection  matrix  W\  maximizes  inter-class  distance, 
clearly  separating  the  three  classes.  Projection  matrix  W2  does  a  poor  job  of  separating  the 
classes  as  evidence  of  the  class  overlap  in  the  projection  space. 

2.3.1.2  GRLVQI 

The  Generalized  Relevance  Learning  from  Vectors  Quantized  Improved  (GRLVQI) 
process  is  implemented  as  described  in  [36].  The  GRLVQI  process  has  the  following 
advantages  over  the  MDA/ML  process:  1)  there  is  no  underlying  assumption  regarding  the 
distribution  of  the  data  2)  it  is  well  suited  for  situations  where  the  number  of  inputs  may 
not  be  consistent  across  classes  and  3)  most  importantly  it  allows  the  ranking  of  individual 
features  according  to  their  ability  of  creating  classification  boundaries  that  minimize  Bayes’ 
risk. 
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Figure  2.2:  MDA/ML  Model  Represen¬ 
tation  for  Nc=  3  Classes. 


Figure  2.3:  Class  MDA/ML  Projection 
onto  Ac-1  Dimensional  Plane 


The  GRLVQI  process  uses  NP= 10  prototype  vectors,  where  each  vector  is  composed 
of  Nf  features,  (See  Table  3.3  for  the  feature  dimensionalities  considered)  to  represent 
a  given  device.  The  GRLVQI  process  as  used  in  this  research  performs  classification  by 
measuring  the  Euclidean  distance  from  a  projected  fingerprint  to  the  prototype  vectors.  The 
projected  fingerprint  is  classified  as  belonging  to  the  class/device  for  which  the  Euclidean 
distance  from  the  projection  fingerprint  to  the  prototype  vector  is  minimized.  Although 
other  distance  measures  exist  (Mahalanobis,  Manhattan  City  Block,  Nearest  Neighbor 
etc.)  Euclidean  distance  is  used  here,  and  has  been  shown  successful  in  previous  research 
[4,  34].  Figure  2.4  is  a  visual  representation  of  prototype  vectors  representing  a  respective 
class/device  with  an  unknown  fingerprint  being  presented  for  classification.  Figure  2.5  the 
shows  relevance  ranking  for  Time-Domain  (TD)  features  for  a  given  fingerprint. 

2.3.2  Verification 

During  classification  DUT  fingerprints  are  aligned  to  a  class  (correctly  or  incorrectly). 
Verification  is  a  one-to-one  “how  much  like”  comparison  with  the  goal  of  determining 
weather  the  unknown  DUT  fingerprints  can  be  verified  as  the  known  device  it  is  being 
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ID  Time  Domain  Feature  Illustration 


Figure  2.4:  GRLVQI  Feature  Space 


1  10  20  30  40  50  60  70 

Unsorted  Feature  Index  # 


Figure  2.5:  GRLVQI  Relevance  Rank¬ 
ings 


aligned  to  during  Classification.  While  a  device  is  classified  according  to  the  class  it  is 
closest  to  using  the  selected  distance  metric,  a  device  is  verified  (authorized  or  rejected) 
based  the  actual  value  of  the  distance  metric. 

This  research  follows  verification  techniques  used  in  [6,  8,  36].  Verification  results 
shown  here  are  presented  as  Receiver  Operating  Characteristic  (ROC)  curves.  Both 
classifiers  presented  in  Section  2.3.1  use  the  same  verification  process  which  is  further 
discussed  in  Section  3.7.2. 
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III.  Methodology 


This  chapter  discusses  the  approach  taken  to  develop  Radio  Frequency  Distinct  Native 
Attribute  (RF-DNA)  fingerprints  used  for  Programmable  Logic  Controller  (PLC)  device 
discrimination.  The  process  is  applied  to  data  collected  at  Oak  Ridge  National  Laboratory 
(ORNL)  using  the  National  Instruments  receiver  platform.  The  collection  process  is  based 
on  an  existing  Air  Force  Institute  of  Technology  (AFIT)  collection  process  [43].  During 
the  signal  collection  phase  at  ORNL  several  receiver  timing  issues  arose  that  demanded 
an  alteration  to  the  existing  collection  process.  The  collection  alteration  resulted  in  fewer 
Region  Of  Interest  (ROI)s  being  collected  compared  to  previous  AFIT  research.  For  this 
reason  the  ROI  extraction  method  was  also  modified  to  ensure  an  adequate  amount  of  ROIs. 
The  process  shown  here  documents  the  process  used  for  collection  at  ORNL  and  reflects 
the  changes  made  from  previous  research  efforts. 


Table  3.1:  Receiver  Collection  Platforms 


AFIT  Collection  Platform 

ORNL  Collection  Platform 

Platform  Manufacturer 

Lecroy 

National  Instruments 

Platform  Model  Number 

WaveMaster 

PXIe-1085  Chassis, 

PXIe-8135  Embedded  Controller 

Platform  Cost 

$127,000.00 

$25,600.00 

3.1  PLC  Device  Description 

This  research  focused  on  applying  the  hardware  discrimination  process  to  10  Allen 
Bradley  SLC-500  PLC  5/02  CPU  module  devices  that  are  collected  against.  The  PLCs  used 
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are  Commercial  Off  The  Shelf  (COTS)  devices  whose  internal  Microcontroller  Unit  (MCU) 
has  comparable  architecture  to  other  COTS  Integrated  Circuits  (IC)  devices  [7,  9].  The 
devices  listed  are  numbered/named  based  on  variable  markings  and  labels  in  the  same 
manner  as  [43],  shown  in  Table  3.2.  One  device,  ZC  was  used  in  initial  collections  [43], 
but  is  not  considered  for  comparison  due  to  operational  difficulties  encountered  during  the 
ORNL  collection. 


Table  3.2:  Component  Under  Test  (CUT)  to  PLC  Identity  ID  Mapping  Based  on  Device 
Labelling  and  Logos  [43]. 


Device  ID 

MCU  Label 

MCU  Logo 

PLC  ID 

Device  1 

NXP 

None 

WQ 

Device  2 

NXP 

None 

WV 

Device  3 

None 

Philips 

KG 

Device4 

None 

Philips 

QI 

Device  5 

Philips 

Philips 

KV 

Device  6 

Philips 

Philips 

OV 

Device7 

Philips 

Philips 

RG 

Device  8 

None 

Philips 

ZC 

Device9 

None 

Philips 

ZZ 

Device  10 

Signetics  &  Intel 

Signetics 

ZA 

The  devices  are  split  into  groups:  Authorized  Devices  and  Rogue  Devices.  Authorized 
Devices  are  PLC  hardware  devices  whose  RF-DNA  fingerprints  are  used  for  model 
development  which  is  discussed  in  Section  3.7.  Rogue  Devices  are  PLC  hardware  devices 
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whose  RF-DNA  fingerprints  are  not  used  during  model  development.  Rogue  Devices  are 
used  only  during  Verification  which  is  further  discussed  in  Section  3.7.2. 

PLC  devices  are  assigned  as  either  an  Authorized  or  Rogue  device  based  on  their  relative 
spectral  intensity  plots,  shown  in  Figure  3.1,  [43].  Authorized  Devices  presented  in  this 
research  are  {WQ,  WV,KV,RG,OV}.  The  Rogue  Devices  presented  in  this  research  are 
{KG,  QI,ZA,ZZ).  Although  labeled  Authorized/  Rogue  for  the  purpose  of  this  research, 
all  devices  are  authentic  Allen  Bradley  PLCs  purchased  through  standard  COTS  channels. 


Figure  3.1:  “Spectral  intensity  plots  generated  as  emission  maximum  PSD  responses  over 
a  20  x  20  uniform  grid  above  the  PLC  MCU  surface.”  [43] 


3.2  RF  Signal  Collection 

3.2.1  PLC  Collection  Configuration 

Each  Device  Under  Test  (DUT)  is  removed  from  it’s  manufactured  housing  so  that  the 
PLC  mainboard  is  completely  exposed.  This  allows  the  RF-probe  to  be  placed  on  top  of 
the  MCU  within  a  given  DUT.  The  exposed  PLC  mainboard  is  placed  onto  a  table  which 
holds  the  RF  probe  and  allows  precise  positioning  of  the  probe  in  three  dimensions.  The 
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PLC  mainboard  is  powered  through  extension  cables  with  the  same  extension  cables  used 
for  all  DUTs. 

There  are  two  collection  platforms  used,  detailed  in  Table  3.1.  The  collection  platforms 
are  configured  in  the  same  method,  with  one  exception.  Inspector  software  is  used  as  the 
instrument  controlling  software  for  the  Lecroy  collection  platform.  Matlab®  is  used  as  the 
instrument  controlling  software  for  the  National  Instruments  collection  platform. 

3.2.2  RF  Probe  Placement 

Because  each  DUT  must  be  connected  and  disconnected,  the  collection  procedure  has 
the  potential  for  error  in  probe  placement.  A  probe  placement  routine  is  implemented  to 
mitigate  repeatability  issues.  The  probe  placement  routine  developed  in  [43]  is  adopted 
into  Matlab®  (replacing  Inspector)  as  the  software  to  control  the  physical  re-positioning  of 
the  probe  for  the  National  Instruments  collection  platform  using  AFIT  generated  control 
functions. 

The  routine  has  two  steps:  1)  Course  Probe  Placement  -  The  probe  is  placed  at  a 
physically  marked  predetermined  position  on  the  DUT  surface  2)  Refined  Probe  Placement 
-  the  probe  is  repositioned  to  the  site  where  Unintentional  Radiated  Emission  (URE)  will 
be  collected  for  the  purpose  of  generating  RF-DNA  fingerprints. 

Once  the  probe  has  been  coarsely  placed  based  on  the  physical  markings,  emissions  are 
collected  at  Nr  =  1 00  locations  on  a  (Dx  =  10)  x  (Dv  =  10)  dimensional  grid  where  the 
grid  size  is  (xm  =  0.75cm)  x  (ym  =  0.75cm).  At  each  grid  location  a  collection  is  taken  of 
the  URE  produced  during  the  execution  of  one  Ladder  Logic  Program  (LLP)  scan.  During 
this  phase  of  probe  re-positioning  the  LLP  being  executed  by  the  DUT  is  referred  to  as  the 
alignment  LLP.  The  alignment  LLP  consists  of  a  known  sequence  of  N0p  =  6  operations: 
{MOV,  SQR,  MOV,  SQR,  MOV,  SQR}. 

To  determine  which  of  the  NL= 100  locations  the  RF  probe  will  return  to  for  further 
collection,  a  previously  collected  and  stored  alignment  reference  signal  x /?[«]  representing 
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a  pristine  alignment  LLP  collection  is  used.  While  there  are  A/,  =  100  alignment  signal 
collections,  there  is  only  one  alignment  reference  signal  xR[n],  xR\ri\  consists  of  Nop  =  2 
operations:  {MOV,  SQR}.  The  alignment  reference  signal  is  empirically  chosen  by  means 
of  a  superior  quality  URE  collection.  The  same  alignment  reference  signal  is  used  in  all 
DUT  alignment  routines. 

The  final  re-positioning  of  the  probe  (refined  probe  placement)  is  determined  by  cross- 
correlating  the  alignment  reference  signal  xR[n]  with  of  the  each  A^/  =  1 00  alignment  signal 
collections.  The  location  yielding  the  highest  correlation  metrics  derived  as  in  [43]  is 
chosen  as  the  refined  probe  position  and  all  further  DUT  URE  collections  are  taken  at 
that  location. 

3.2.3  Sampling  and  Triggering 

The  frequency  of  interest  for  the  generation  of  RF-DNA  fingerprints  is  fc= 55.5  MHz. 
The  observed  clock  frequency  of  the  Allen  Bradley  PLC  MCU  is  /),«.=  18.5  MHz.  The 
strongest  component  of  the  observed  clock  frequency  is  the  third  clock  harmonic  centered 
at  /= 55.5  MHz.  To  prevent  aliasing  during  signal  collection  an  in-line  Low  Pass  Filter 
(LPF)  is  used  with  a  cutoff  frequency  of  /co=81.0  MHz. 

All  DUT  RF  emissions  are  collected  at  the  sampling  frequency  rate,  fs= 250  MSps  using 
a  near  field  probe  with  baseband  bandwidth  Wbb= 500  MHz.  The  existing  AFIT  data  set 
was  collected  using  the  LeCroy  collection  platform,  and  the  ORNL  dataset  was  collected 
using  the  National  Instruments  collection  platform  as  shown  in  Table  3.1.  The  collected 
emissions  are  stored  sequentially  as  8  bit  integer  values  representing  the  measured  voltage 
level  of  the  collected  signal  at  evenly  spaced  time  intervals. 

For  the  National  Instruments  collection  platform  using  Matlab®,  two  triggers  are 
supplied  to  the  collection  platform  to  instantiate  a  signal  collection.  Both  triggers  must  be 
present  for  a  signal  collection  to  occur.  The  first  trigger  is  sent  from  a  Laptop  being  used  as 
an  instrument  controller,  (controlling  the  RF-probe  placement  and  the  collection  platform) 
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to  the  collection  platform  indicating  that  the  RF-probe  is  in  position  for  a  collection.  This 
is  done  so  that  the  physical  movement  of  the  RF-probe  is  synchronized  with  the  collection 
platform  and  collections  are  not  taken  while  the  probe  is  moving  between  one  of  the 
Nl=100  locations.  The  second  trigger  is  a  threshold  value,  based  on  the  voltage  across 
a  Light  Emitting  Diode  (LED)  on  the  DUT,  prior  to  the  first  MOV  operation  of  an  LLP. 
The  LED  voltage  is  toggled  as  a  square  wave  with  an  approximate  duty  cycle  of  50%  equal 
in  length  to  the  LLP  scan  time.  This  trigger  indicated  the  start  of  an  LLP  scan. 

When  both  triggers  are  present,  indicating  the  probe  is  in  position  (first  trigger)  and  an 
LLP  scan  has  instantiated  (second  trigger)  a  tSiG= 5  ms  collection  is  taken.  A  tSiG= 5  ms 
collection  is  taken  to  ensure  the  entire  URE  produced  from  the  execution  of  an  LLP  scan  is 
collected  as  the  LLP  scan  is  approximately  tLLP= 3  ms.  This  triggering  process  is  used  for 
each  Nl  =100  locations,  as  well  as  the  subsequent  refined  probe  placement  position. 

3.3  Post  Collection  Processing 

After  signal  collection,  post-collection  processing  is  done  using  Matlab®.  The 
collections  are  read  into  Matlab®  from  the  binary  file  and  converted  to  type  double  for  use 
with  Matlab®  filtering  functions.  The  signals  are  then  processed  according  to  the  following 
steps:  1)  digital  bandpass  filtering  2)  down-conversion  to  an  intermediate  frequency  3) 
down- sampling  4)  SNR  scaling. 

1.  Bandpass  Filtering  -  The  signals  are  bandpass  filtered  using  a  digital  8f/l-order 
Butterworth  bandpass  filter  with  a  center  frequency  of  fBp= 55.5  MHz  and  -3.0  dB 
bandwidth  of  WBp=1.0  MHz.  This  is  done  using  the  built  in  Matlab®  function 
butter []  to  generate  filter  coefficients  and  filtfilt[]  to  perform  the  actual 
filtering.  The  magnitude  response  of  the  filter  is  shown  in  Figure  3.2. 

2.  Downconversion  -  After  bandpass  filtering,  the  signals  are  downconverted  from 
the  range  of  /  e  [55.0,56.0]  MHz  to  the  range  of  /  e  [1.0, 3.0]  MHz.  Once 
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Figure  3.2:  Magnitude  Response  of  8,/;  order  bandpass  Butterworth  Filter  [43]. 


downconverted  the  signals  are  then  digitally  filtered  with  a  LPF.  The  cutoff  frequency 
for  the  LPF  is  fco  =  3.5  MHz. 

3.  Downsampling  -  After  filtering  and  downconversion  the  signals  are  downsampled 
by  a  factor  of  Ds  20,  reducing  the  number  of  samples  to  yield  an  effective  sample 
rate  of  fs= 12.5  MSps.  Downsampling  is  accomplished  by  selecting  the  first 
element/sample  of  a  signal,  and  henceforth  every  20',!  element/sample,  where 
unselected  elements/samples  are  discarded. 

4.  SNR  Scaling  -  independent  Additive  White  Gaussian  Noise  (AWGN)  realizations 
are  added  to  the  post-processed  signals  to  simulate  a  range  of  channel  Signal  to 
Noise  Ratio  (SNR),  .  This  is  done  to  reduce  the  number  of  collections  that  would 
otherwise  be  needed  to  evaluate  performance  under  degraded  conditions.  The  range 
of  SNR  values  presented  in  this  research  is,  SNR  6  [-30  :  30]  dB  in  SNRstep= 5  dB 
increments.  For  each  signal  collected,  at  each  SNR  considered,  Nnz=  10  noise 
realizations  are  simulated.  It  is  important  to  note  that  although  SNR  scaling  is 
considered  to  be  digital  post-processing  it  occurs  after  ROI  Extraction,  discussed 
in  the  next  section. 
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3.4  ROI  Extraction 


As  mentioned  previously  an  LLP  execution  takes  approximately  tu.p= 3  ms  with  the 
actual  ROI  spanning  tROi=  1.5  ms.  The  signal  collection  platform  collects  for  tsig= 5  ms. 
This  is  done  to  ensure  the  entire  ROI  is  captured  in  the  collected  waveform.  ROI  extraction 
process  isolates  the  ROI  from  the  unwanted  part  of  the  signal  collection.  ROI  extraction 
occurs  after  digital  signal  processing,  therefore  all  signal  collections  described  henceforth 
are  assumed  to  have  been  digitally  post-processed  according  to  Section  3.3  with  the 
exception  of  added  noise  realizations. 

Consider  a  given  collection  sequence  xc[n ]  =  .rc[l]  +  xc[2]+. . .  +xc[n],  n=l,2  ...Nc, 
where  Nc  is  the  last  collection  sample.  Also  consider  following  sequences  xas  [ n I  and 
xES  \n\.  xAs  \n  I  represents  the  alignment  start ,  i.e.  discrete  samples  of  the  LLP  operations 
{MOV.SQR}  (those  operations  that  begin  every  scan  of  the  LLP  that  is  being  executed). 
xEs  [»]  represents  the  alignment  end ,  the  operations  {SQR.MOV}  (those  operations  that  end 
each  LLP). 

The  start  of  an  ROI  is  determined  by  cross-correlating  the  collected  signal  sequence 
,v'c[h]  with  the  alignment  start  sequence  xas  [ » I  •  The  end  of  an  ROI  is  determined  by  cross- 
correlating  x£s[/7]  with  the  signal  collection  xc\ri\.  The  ROI  length  is  then  estimated  by 
finding  the  difference  between  lag  values  for  the  corresponding  maximum  cross-correlation 
values  for  the  start  time  CMs  and  end  time  CMe- 

The  mean  and  standard  deviation  of  the  estimated  ROI  lengths  are  calculated,  nROILen 
and  crROILen  respectively.  A  threshold  value  of  Umiun  +  1.5 crRoILen  is  established  and  any 
ROIs  exceeding  the  threshold  are  discarded.  As  previously  mentioned  in  Chapter  2  and 
shown  in  [43],  unaccounted  CPU  operations  can  occur  during  the  execution  of  the  LLP. 
The  extra  CPU  operations  are  unwanted  and  render  an  ROI  unusable.  ROIs  that  contain 
extra  operations  (those  ROIs  the  exceed  the  length  threshold)  are  therefore  discarded. 
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Figure  3.3:  Region  Of  Interest  Extraction 


The  remaining  ROIs  are  sorted  in  descending  order  by  the  mean  of  their  maximum 
cross-correlation  values  {Cms-Cme}-  From  this  sorted  set  (first  element  in  the  set  has  the 
highest  mean  of  { CM5,CMe }),  the  top  250  ROIs  are  chosen  for  Fingerprint  generation.  The 
remaining  ROIs  are  not  considered  further.  Signal  collection  at  ORNF  included  multiple 
collections  over  the  course  of  two  consecutive  days.  Ultimately,  500  ROIs  are  used  for 
fingerprint  generation  (250  ROIs  from  the  two  independent  collections). 

After  selecting  the  best  quality  ROIs  based  on  the  correlation  metrics  described,  the  ROIs 
must  be  extracted  from  the  collected  signal.  This  is  done  by  using  the  sample  index  of  the 
maximum  correlation  start  and  end  times. 

3.5  Fingerprint  Generation 

RF-DNA  fingerprint  generation  was  implemented  in  accordance  with  previous  AFIT 
RF-DNA  fingerprinting  research  [4,  9,  36,  43].  The  process  has  been  applied  to  sequences 
representing  Time-Domain  (TD),  Frequency  Domain  (FD)  and  Time-Frequency  domain 
data  sets  [4,  9,  17,  36,  43,  49].  For  the  purpose  of  this  research  only  TD  features  are 
considered. 

For  the  complex  signal  x[n\  =  xre[n\  +  xim[n\,  the  instantaneous  TD  responses  Amplitude 
a[n ],  Phase  (p\n\  and  Frequency  f\n\  are  given  by, 
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Arbitrary  Feature  Sequence 


ha  1  c'«j’  1 


Figure  3.4:  Abstract  representation  of  RF-DNA  fingerprint  formation  for  an  arbitrary 
sequence  divided  into  N/s>  subregions  [49] 


a[n\  =  V xre{n ]2  +  xim[n]2, 
Xim\n\ 


(p[n\  =  tan  1 


Xre\n] 

m  -  h 


,  xre[n\  ±  0, 

dcf)[n] 
dn 


(3.1) 

(3.2) 

(3.3) 


The  following  steps  give  an  overview  of  the  RF-DNA  fingerprinting  process  that  is 
implemented  for  fingerprints  generated  from  URE  using  the  instantaneous  TD  features 
defined  in  Eqns.  3. 1-3.3. 

1.  A  selected  ROI  is  divided  into  NR  equal  contiguous  time-domain  sub-regions. 

2.  Within  each  subregion  the  mean  fi  is  calculated  and  subtracted  from  all  subregion 
samples  to  minimize  the  impact  of  collection  bias. 


3.  The  Nfeat= 3  instantaneous  TD  responses  (Amp  a[n],  Phz  cp[n],  Frq  /[//]  )  are  found 
for  each  subregion. 
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4.  The  Nstat=4  statistical  attributes  (standard  deviation  cr,  variance  cr2,  skewness  y,  and 
kurtosis  k)  are  found  for  each  subregion  and  each  TD  response. 

5.  The  resulting  statistical  attributes  are  concatenated,  and  comprise  an  individual  RF- 
DNA  fingerprint  that  represents  one  ROI  noise  realization  for  a  given  SNR  and  given 
device. 

6.  The  process  is  repeated  for  all  ROIs  across  all  noise  realizations,  all  SNRs,  and  all 
devices. 

In  total  there  are  NF  =  65000  RF-DNA  fingerprints  per  PLC  device. 

Nf  =  Nroi  x  Nnz  x  Ns  nr  (3.4) 

3.6  Feature  Set  Dimensional  Reduction 

The  process  described  in  Section  3.5  is  implemented  using  Nr= 12  subregions  as  well 
as  calculating  statistics  over  the  entire  ROI.  The  full  dimensionality  of  a  given  fingerprint 
is  therefore  ND = 1 5  6 . 


ND  =  (NR  +  l)xNfeatxNstat  (3.5) 

Reducing  the  fingerprint  dimensionality  is  done  by  considering  a  subset  of  the  full 
dimensional  features.  Qualitatively  and  Quantitatively  selected  subsets  are  considered. 
Table  3.3  details  the  dimensionality  of  the  fingerprint  sets  used  for  classification  and 
verification  for  the  results  presented  in  Chapter  4. 

Of  interest  to  this  research  is  reducing  the  dimensionality  of  the  feature  set.  Dimensional 
reduction  is  explored  to  enhance  experimental-to-operational  transition  potential  of  RF- 
DNA  fingerprinting  [36]. 
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Table  3.3:  Feature  sets  used  for  Classification  and  Verification  in  GRLVQI  &  MDA/ML 


Type 

Feature  Set 

Number  of  Features 

Number  of  Fingerprints 

Full  Dimensional 

Full 

156 

500 

Qualitative  DRA 

Amplitude 

52 

500 

Qualitative  DRA 

Phase 

52 

500 

Qualitative  DRA 

Frequency 

52 

500 

Quantitative  DRA 

Top  33% 

52 

500 

Quantitative  DRA 

Top  10% 

16 

500 

3.6.1  Qualitative 

Qualitative  feature  sets  refer  sets  of  fingerprints  whose  features  are  qualitatively 
selected  and  are  solely  composed  of  either  Amplitude,  Phase,  or  Frequency  statistics. 
The  same  Nr  subregions  are  used  to  calculate  statistics,  however  the  fingerprints  in  each 
qualitative  set  are  of  only  one  time  domain  response.  The  fingerprints  are  one  third  of 
the  size  of  the  full  dimensional  feature  set  fingerprints.  The  number  of  fingerprints  in  all 
feature  sets  is  constant. 

3.6.2  Quantitative 

Features  are  ranked  in  descending  order  according  to  their  relevance  ranking 
determined  by  the  GRLVQI  process.  Quantitative  feature  sets  refer  to  fingerprints  whose 
features  are  a  subset  of  the  full  dimensional  feature  set  that  have  been  selected  based  on 
a  relevance  ranking.  The  Top  33%  feature  set  is  composed  of  the  top  ranked  52  of  156 
features.  Those  features  can  be  of  any  time  domain  response.  Likewise  the  Top  10% 
feature  set  is  composed  of  the  top  ranked  16  features.  The  Top  10%  feature  set  is  contained 
in  the  Top  33%  feature  set. 
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3.7  Device  Discrimination 


3.7.1  Classification 

Following  the  formation  of  RF-DNA  fingerprints,  PLC  hardware  discrimination 
is  performed.  Two  methods  for  PLC  device  classification  are  considered  in  this 
research.  Generalized  Relevance  Learning  from  Vectors  Quantized  Improved  (GRLVQI) 
and  Multiple  Discriminant  Analysis  Maximum  Likelihood  (MDA/ML).  Although 
their  respective  internal  mechanisms  for  hardware  classification  are  different,  the  two 
independent  processes  make  use  of  the  same  approach  for  both  classification  and 
verification.  Both  classification  processes  use  RF-DNA  fingerprints  to  identify  a  given 
PLC  hardware  device.  RF-DNA  fingerprints  (generated  as  described  in  Section  3.5)  are 
divided  into  two  equal  sized  sets;  Training  fingerprints  (xTN(AtA)  and  Testing  fingerprints 
(xtst [«]).  The  fingerprints  are  divided  based  on  an  interleaved  pattern  (odd  and  even 
number  indices).  For  the  results  shown  in  Chapter  4,  the  total  number  of  fingerprints  used 
is  N g— 500  (Ntng=250  and  Ntst=250'). 

Training/Validation  -  The  xTNG  \  n  ]  set  of  fingerprints  are  used  by  the  GRLVQI 
and  MDA/ML  processes  to  develop  a  device  classification  model.  The  set  of  xTNa\fi\ 
fingerprints  are  divided  into  k  segments  following  a  kfold  partitioning  process  [36].  The 
GRLVQI  and  MDA/ML  processes  use  k- 1  segments  to  develop  a  classification  model, 
where  the  kth  segment  is  held  out  and  is  used  after  the  model  is  developed  to  perform 
model -validation.  The  kth  segment  is  introduced  after  model  development  to  assess 
the  performance  of  that  model  in  correctly  identifying  a  given  devices  fingerprints.  All 
permutations  of  the  k-fold  model  development  and  subsequent  model -validation  are  carried 
out  in  turn.  After  all  k-fold  model  development  permutations,  the  model  yielding  the  best 
validation  results,  (percent  correct  classification)  is  chosen.  Training/Validation  is  repeated 
independently  for  each  SNR  considered. 
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Testing  -  Testing  is  the  phase  of  model  development  where  previously  unseen 
fingerprints  are  introduced  to  the  model  that  is  selected  during  the  validation  phase.  Testing 
assesses  the  model’s  ability  to  correctly  identify  devices.  The  x Tsr[n\  set  of  fingerprints 
are  held  out  of  model  development;  xTS  7- [  n  ]  fingerprints  represent  previously  unseen  data 
to  assess  the  performance  of  the  classification  model. 

3.7.2  Verification 

Verification  is  a  method  of  examination  to  determine  how  well  a  given  device’s  xTS  r  \ n  I 
fingerprints  resemble  what  they  are  being  classified  as.  Verification  allows  a  one-to-one 
comparison  based  on  a  measure  of  similarity  test  statistic  zv-  The  test  statistic  used  in  both 
MDA/ML  and  GRLVQI  is  Euclidean  Distance  as  derived  in  [9]  [36]  respectively. 

A  Probability  Mass  Function  (PMF)  of  z,v  is  constructed  for  each  device/class  .  A  device’s 
classified  identity  is  then  verified  (correctly  or  incorrectly)  by  a  binary  decision  against 
a  threshold  value  tv.  If  the  test  statistic  is  below  the  threshold  the  device  is  declared 
Authorized  (correctly  or  incorrectly).  Conversely  a  test  statistic  over  the  threshold  is 
declared  Rogue.  The  threshold  value  tv  as  used  in  this  research  is  implemented  as  described 
in  [36]. 


xTsr[n\  -»  zv[n\  <  tv 

:  Authorized 

(3.6) 

xtstM  zv[n\  >  tv 

:  Rogue 

(3.7) 

3.8  Performance  Evaluation 

Classification  performance  as  described  in  Section  2.3.1  is  evaluated  at  an  arbitrary 
baseline  performance  of  90%  correct  classification  for  a  given  SNR.  The  CAve= 90% 
baseline  performance  metric  has  been  used  in  previous  AFIT  research  efforts  [9,  36,  43]. 

The  possible  verification  outcomes  are  shown  in  Table  3.4.  Verification  performance 
is  assessed  using  Receiver  Operating  Characteristic  (ROC)  curve  plots.  There  are  two 
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Table  3.4:  Authorized  vs.  Rogue  Identification 


Actual 

Claimed 

Declared 

Outcome 

Authorized 

Authorized 

Authorized 

True  Authorized  Accept 

Authorized 

Authorized 

Rogue 

False  Authorized  Reject 

Rogue 

Authorized 

Authorized 

False  Rogue  Accept 

Rogue 

Authorized 

Rogue 

True  Rogue  Reject 

types  of  ROC  curves  that  are  presented  in  Chapter  4,  Authorized  Device  Identification  and 
Rogue  Device  Rejection.  Authorized  Device  Identification  is  a  plot  of  False  Verification 
Rate  (FVR)  vs.  True  Verification  Rate  (TVR).  Rogue  Device  Rejection  is  a  plot  of  Rogue 
Accept  Rate  (RAR)  vs.  TVR. 


FVR  = 

Y  FalseAuthorizedRe  ject 

(3.8) 

Y  T  rue  Authorized  Accept +Y  FalseAuthorizedRe  ject 

RAR  = 

Y  FalseRogueAccept 

Y  T  rue  Rogue  Re  ject+Y  FalseRogueAccept 

(3.9) 

TVR  = 

Y  T rueAuthorizedCount 

Y  T rueAuthorizedCount+Y  FalseAuthorizedCount 

(3.10) 

The  Authorized  ID  plots  show  how  much  a  given  Authorized  Device  looks  like  itself 
when  compared  to  the  other  Authorized  Devices.  The  Rogue  Device  Identification  plots 
shows  how  well  a  given  classification  model  can  correctly  reject  rogue  devices  and  correctly 
accept  authorized  devices. 
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IV.  Results 


This  chapter  details  the  results  of  Programmable  Logic  Controller  (PLC)  device  clas¬ 
sification  and  verification  processes  using  the  Multiple  Discriminant  Analysis  Maximum 
Likelihood  (MDA/ML)  and  Generalized  Relevance  Learning  from  Vectors  Quantized  Im¬ 
proved  (GRLVQI)  classifiers  as  described  in  Section  2.3.1  using  Radio  Frequency  Dis¬ 
tinct  Native  Attribute  (RF-DNA)  fingerprints  developed  independently  from  the  Lecroy  and 
National  Instruments  (NI)  collection  platform  data  described  in  Table  3.1  are  used  as  inputs 
for  the  classification  and  verification  processes.  Section  4. 1  shows  results  for  the  Lecroy 
platform  fingerprints,  and  Section  4.2  shows  results  for  the  NI  platform  fingerprints.  For 
each  collection  platform  there  are  6  feature  sets  considered  as  listed  in  Table  3.3,  where  all 
Dimensional  Reduction  Analysis  (DRA)  Feature  sets  are  subsets  of  the  Full  Dimensional 
feature  set. 

The  RF-DNA  fingerprints  are  developed  from  Unintentional  Radiated  Emission  (URE) 
signal  collections  taken  from  Ndev-  10  PLC  hardware  devices  where  each  device  has 
NF= 500  fingerprints.  As  mentioned  in  Chapter  3  device  ZC  is  not  considered  due  to 
unpredictable  device  operation.  The  fingerprints  are  generated  from  Time-Domain  (TD) 
signal  responses  as  described  in  Section  3.5.  Fingerprints  in  the  range  of  Signal  to  Noise 
Ratio  (SNR)e[-30:5:30]  dB  are  used  for  comparison  of  MDA/ML  and  GRLVQI  classifiers 
as  well  as  to  compare  the  Lecroy  and  NI  collection  platforms.  Nnz=  1 0  independent  Additive 
White  Gaussian  Noise  (AWGN)  realizations  for  each  fingerprint  at  each  SNR  are  used  to 
simulate  channel  effects  over  the  SNR  range.  For  the  purpose  of  comparing  classifiers, 
feature  sets,  and  collection  platforms,  gain  is  used  to  specify  the  difference  of  SNR  at 
which  respective  performances  are  equivalent. 
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4.1  Expansion  of  Lecroy  Platform  RF-DNA  Fingerprinting  Results 

The  Lecroy  collection  platform  data  set  was  used  in  prior  Air  Force  Institute  of 
Technology  (AFIT)  research  [7,  9,  43].  The  existing  data  is  used  in  this  research  to  expand 
upon  the  previous  device  discrimination  results  i.e.  results  from  the  Lecroy  collection 
platform  shown,  are  from  previously  existing  signal  collections.  The  only  new  signal 
collections  presented  are  signal  collections  taken  using  the  NI  collection  platform.  The 
expansion  of  Lecroy  platform  collection  results  includes  analysis  of  feature  dimensional 
reduction  as  well  as  the  use  of  the  MDA/ML  classifier. 

4.1.1  Full  Dimensional 

Figure  4.1(a)(b)  shows  classification  results  using  the  full  dimensional  feature  set 
AV=156,  over  the  SNR  range  of  [-30:30]  dB  in  5  dB  increments.  Figure  4.1(a)  shows 
the  MDA/ML  classifier  achieves  a  cross-device  average  CAVE= 90.0%  correct  classification 
for  SNR>6.5  dB.  Figure  4.1(b)  shows  the  GRLVQI  classifier  achieves  the  CAVE=  90.0% 
correct  benchmark  for  SNR>11  dB.  The  MDA/ML  classifier  is  outperforms  the  GRLVQI 
classifier  with  a  GSNR~4.5  dB  gain  relative  to  the  GRLVQI  classifier. 

4.1.2  Dimensional  Reduction 

Qualitative  feature  sets  are  dimensional  reduced  by  using  features  generated  from 
one  of  the  Nfea,= 3  TD  signal  responses  described  in  Section  3.5.  The  selected  features  are 
a  subset  of  the  full  dimensional  feature  set  where  two  TD  signal  response  features  types 
have  been  removed. 

Consistent  across  the  Lecroy  platform  Qualitative  DRA  feature  sets,  the  MDA/ML 
classifier  outperforms  the  GRLVQI  classifier.  The  MDA/ML  classifier  has  a  Gsnr~5  dB 
gain  in  the  Amplitude  set,  a  GSnr~3  dB  gain  in  the  Phase  set,  and  GSnr~ 2.5  dB  gain  for  the 
Frequency  set  against  the  respective  GRLVQI  Qualitative  DRA  feature  set  when  comparing 
benchmark  performance  (90.0%  correct  classification). 
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(a)  MDA/ML  Full  feature  Testing  Results 


(b)  GRLVQI  Full  Feature  Testing  Results 


Figure  4.1:  Full  Dimensional  Testing  Results  for  MDA/ML  and  GRLVQI  Lecroy 
Collection  Platform 


Of  the  three  Qualitative  DRA  feature  sets  the  Amplitude  feature  set  yields  best 
classification  performance  for  both  the  MDA/ML  and  GRLVQI  classifiers.  Using  the 
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(a)  MDA/ML  Testing  Results 


(b)  GRLVQI  Testing  Results 


Figure  4.2:  Lecroy  Platform  DRA  Testing  Results 


Amplitude  Feature  results  in  a  benchmark  performance  gain  of  Gsnr~ 3  dB  when  compared 
to  both  the  Phase  and  Frequency  DRA  feature  sets  in  both  receiver  platforms. 
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Quantitative  feature  sets,  like  the  Qualitative  feature  sets,  are  subsets  of  the  Full 
dimensional  feature  set.  The  Quantitative  feature  sets  are  constructed  by  examining  the 
relevance  rankings  generated  by  the  GRLVQI  classifier.  Features  are  assigned  a  weighted 
value  (relevance  rank)  according  to  how  well  they  impact  classification  performance  [36]. 
There  are  two  Quantitative  feature  sets,  Top  33%  and  Top  10%.  As  previously  mentioned 
both  feature  sets  are  contained  in  the  Full  dimensional  feature  set;  the  Top  10%  feature 
set  is  a  subset  of  the  Top  33%  feature  set.  The  Quantitative  feature  sets  are  the  respective 
percentage  of  the  highest  ranked  features.  Although  the  MDA/ML  classifier  does  not  have 
the  inherent  ability  to  produce  relevance  rankings,  Quantitative  feature  sets  constructed 
from  the  GRLVQI  classifier  are  used  by  the  MDA/ML.  Fig.  4.3  shows  an  overlay  of  the 
relevance  rankings  for  the  Full  dimensional  feature  set  for  both  the  Lecroy  and  NI  collection 
platforms. 

The  Top  33%  feature  set  outperformed  the  Top  10%  feature  set  in  MDA/ML 
classification  by  GSnr~ 2  dB.  However  for  GRLVQI  classification  the  Full,  Top  10%  and 
Top  33%  feature  sets  are  statistically  equivalent  using  95%  confidence  intervals. 

4.2  National  Instruments  Platform  RF-DNA  Fingerprinting  Results 
4.2.1  Full  Dimensional 

Fig.  4.4  shows  the  classification  results  using  the  Full  dimensional  feature  set 
NF= 156  for  collections  taken  with  the  NI  platform  over  the  SNR  range  of  [-30:30]  dB 
in  S NRsteP= 5  dB  increments.  Fig.  4.4(a)  shows  the  MDA/ML  classifier  achieves  a  cross¬ 
device  average  CAy£-=90.0%  correct  classification  for  SNR>16.5  dB.  Fig.  4.4(b)  shows 
the  GRLVQI  classifier  achieves  the  Cave- 90.0%  correct  benchmark  for  SNR>17  dB.  The 
classifiers  here  achieve  nearly  the  same  performance. 
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Figure  4.3:  Relevance  Rankings  for  Lecroy  and  NI  Collection  Platforms 


4.2.2  Dimensional  Reduction 

Qualitative  results  using  dimensionally  reduced  feature  sets  of  NF=52  are  shown 
in  Fig.  4.5(a)(b).  Consistent  across  the  NI  Qualitative  DRA  feature  sets,  the  GRLVQI 
classifier  outperforms  the  MDA/ML  classifier.  This  is  opposite  of  classifier  performance  in 
Lecroy  platform  collection.  The  GRLVQI  classifier  has  a  GSnr~ 4  dB  gain  in  the  Amplitude 
set.  The  MDA/ML  classifier  does  not  meet  the  benchmark  in  the  Phase  feature  set  while  the 
GRLVQI  classifier  reaches  the  benchmark  for  SNR>20  dB.  The  GRLVQI  classifier  sees  a 
Gsnr~2.5  dB  gain  in  the  Frequency  set  against  the  MDA/ML  classifier. 

The  Amplitude  Feature  set  yields  best  Qualitative  DRA  classification  performance  for 
each  classifier.  Using  the  Amplitude  Feature  set  results  in  a  benchmark  performance  gain 
of  >7  dB  when  compared  to  both  the  Phase  and  Frequency  DRA  feature  sets. 
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(a)  MDA/ML  Full  Dimensional  Testing  Results 


(b)  GRLVQI  Full  Dimensional  Testing  Results 


Figure  4.4:  National  Instruments  Full  Dimensional  Testing  Results 


Quantitative  The  Top  33%  feature  set  outperformed  the  Top  10%  feature  set  in 
MDA/ML  classification  by  GSnr~ 7  dB.  The  opposite  is  true  for  GRLVQI  classification 
where  the  Top  10%  feature  set  outperformed  the  Top  33%  feature  set  by  less  than 
Gsnr~ 3  dB.  This  is  evident  in  both  receiver  platforms  for  the  GRLVQI  classifier.  The 
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(a)  MDA/ML  Testing  Results 


(b)  GRLVQI  Testing  Results 


Figure  4.5:  National  Instruments  Platform  DRA  Testing  Results 


GRLVQI  classifier  is  a  Machine  Learning  Neural  Network.  Due  to  it’s  nature  of 
model  development,  it  is  possible  that  the  classifier  suffers  from  overlearning  [52]  the 
characteristics  for  a  given  device’s  set  of  fingerprints.  By  using  a  model  with  less  features, 
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the  model  is  more  robust  in  characterizing  device  fingerprints  in  turn  yielding  better 
classification  performance. 

4.3  Device  Verification 

Device  ID  verification  enables  a  one-to-one  ”how  much  like”  device  comparison. 
Results  are  shown  for  5  Authorized  devices  [WQ,  WV,KV,RG,  OV}  and  4  Rogue  devices 
{KG,  QI,ZA,ZZ}.  Devices  were  deemed  Authorized  or  Rogue  based  upon  their  relative 
spectral  emission  intensity  plots  [43].  Receiver  Operating  Characteristic  (ROC)  curves  are 
presented  for  the  Full,  and  Top  10%  feature  sets  for  both  MDA/ML  and  GRLVQI  classifiers 
as  well  as  both  Lecroy  and  the  NI  collection  platforms.  The  Full  and  Top  10%  feature 
sets  were  chosen  for  presentation  as  they  represent  the  extremes  of  feature  dimensional 
reduction.  In  the  interest  of  space,  the  other  feature  set  verification  plots  are  presented  in 
the  Appendix.  The  ROC  curves  are  evaluated  at  the  lowest  value  SNR  where  performance 
meets  the  arbitrary  90%  average  correct  classification  benchmark  performance  for  the  Full 
dimensional  feature  set.  The  corresponding  SNR  for  the  Lecroy  collection  platform  is 
S  NR=  10  dB  and  the  corresponding  SNR  for  the  NI  platform  is  S  NR= 20  dB.  Verification 
results  are  shown  at  these  SNRs  for  the  two  collection  platforms  respectively.  The  test 
statistic  used  as  a  measure  of  similarity  is  Euclidean  Distance  for  all  ROC  curves. 

4.3.1  Authorized  Device  Identification 

One  aspect  of  verification  is  to  verify  the  identity  of  a  known  authorized  device.  This  is 
ability  is  assessed  by  comparing  how  similar  the  authorized  devices  resemble  each  other. 
The  Equal  Error  Rate  (EER)  as  described  in  Section  3.7.2  is  used  as  the  performance 
criteria.  Results  in  Fig.  4.6(a)(b)  shows  results  for  both  receiver  platforms  using  the 
MDA/ML  classifier.  The  MDA/ML  classifier  outperforms  the  GRLVQI  classifier  for  both 
receiver  platforms  using  the  Full  Dimensional  set.  All  devices  exceeded  the  EER  of  True 
Verification  Rate  (TVR)>90%  and  False  Verification  Rate  (FVR)<10%  for  the  Lecroy 
collection  platform,  3  of  5  devices  met  the  EER  for  the  NI  collection  platform. 
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Verification  Rate  (TVR)  ^ — N  True  Verification  Rate  (TVR) 


a)  Lecroy  MDA/ML  SNR= 10  dB 


(b)  National  Instruments  MDAML 
SNR= 20  dB 


Figure  4.6:  Full  Dimensional:  Authorized  ID  Verification  Results  using  MDA/ML 
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(a)  Lecroy  GRLVQI  SNR=10  dB 


(b)  National  Instruments  GRLVQI 
SNR= 20  dB 


Figure  4.7:  Full  Dimensional:  Authorized  ID  Verification  Results  using  GRLVQI 
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Fig.  4.7(a)(b)  shows  results  for  both  receiver  platforms  using  the  GRLVQI  classifier, 
with  3  of  5  devices  meeting  the  EER  <10%  benchmark  for  the  Lecroy  platform  and  2  of  5 
meeting  the  EER<10%  benchmark  for  the  NI  collection  platform. 

Fig.  4.8(a)(b)  shows  results  for  the  Top  10%  feature  set  using  the  MDA/ML  classifier, 
and  Fig.  4.9  shows  results  for  the  GRLVQI  classifier.  Device  5  does  not  meet  the  EER 
benchmark  for  any  of  the  results  presented  using  the  Top  10%  feature  set  for  the  verification 
of  Authorized  ID.  Device  2  consistently  fails  meet  the  EER  benchmark  for  both  classifiers 
for  the  Lecroy  collection  platform. 


(b)  National  Instruments  MDA/ML 

(a)  Lecroy  MDA/ML  SNR= 10  dB 

SNR= 20  dB 

Figure  4.8:  Top  10%:  Authorized  ID  Verification  Results  using  MDA/ML 


4.3.2  Rogue  Device  Identification 

For  Rogue  Device  analysis  the  same  thresholding  procedure  used  to  generate  the  ROC 
curves  for  verification  of  Authorized  Device  ID  is  used  to  generate  ROC  curves  for  Rogue 
Device  Rejection  as  is  implemented  in  Section  3.7.2.  Correctly  authorizing  a  known  device 
is  only  one  part  of  the  device  ID  verification.  Rogue  devices,  devices  whose  fingerprints 
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False  Verification  Rate  (FVR) 


(a)  Lecroy  GRLVQI  SNR=10  dB 


False  Verification  Rate  (FVR) 

(b)  National  Instruments  GRLVQI 
SNR= 20  dB 


Figure  4.9:  Top  10%:  Authorized  ID  Verification  Results  using  GRLVQI 


have  not  yet  been  seen  and  are  not  represented  in  the  classification  models,  must  also  be 
considered  in  verification  solution.  Rogue  Device  Identification  measures  “how  much  like” 
a  given  Rogue  Device  resembles  each  of  the  Authorized  devices.  This  analysis  simulates 
a  Rogue  device  presenting  bit-level  credentials  claiming  to  be  a  known  authorized  device 
and  presenting  itself  for  Device  ID  verification. 


4.4  Cross  Receiver  Validation 

By  implementing  the  same  collection  process  on  two  receiver  collections  platforms 
and  allows  direct  comparison  between  the  results.  Although  the  Lecroy  collection  platform 
achieves  the  Cave- 90%  baseline  performance  with  a  gain  of  GSnr~  10  dB  over  the  NI 
platform,  it  should  be  noted  that  AWGN  is  added  to  the  collected  signals  to  degrade  to 
meet  the  desired  performance  level.  At  the  collected  SNR  (i.e.  the  absence  of  simulated 
AWGN)  the  NI  collection  platform  is  able  to  achieve  results  of  100%  correct  classification 
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(a)  Lecroy  MDA/ML  SNR= 10  dB 

Figure  4.10:  Full  Dimensional:  Rogue 
MDA/ML 


(b)  National  Instruments  MDA/ML 
SNR= 20  dB 

Device  Rejection  Verification  Results  using 


(a)  Lecroy  GRLVQI  SNR=  10  dB 

Figure  4.11:  Full  Dimensional:  Rogue 
GRLVQI 


(b)  National  Instruments  GRLVQI 
SNR= 20  dB 

Device  Rejection  Verification  Results  using 
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(a)  Lecroy  MDA/ML  SNR= 10  dB 


(b)  National  Instruments  MDA/ML 
SNR= 20  dB 


Figure  4.12:  Top  10%:  Rogue  Device  Rejection  Verification  Results  using  MDA/ML 


(a)  Lecroy  GRLVQI  SNR=  10  dB 


(b)  National  Instruments  GRLVQI 
SNR= 20  dB 


Figure  4.13:  Top  10%:  Rogue  Device  Rejection  Verification  Results  using  GRLVQI 


for  the  Full  dimensional  feature  set.  Applying  the  verification  process  at  the  collected  SNR, 
all  devices  meet  the  EER  for  both  classifiers,  for  both  Authorized  Device  ID  Fig.  4.14  and 
Rogue  Device  Rejection  Fig.  4.15. 
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(a)  National  Instruments  MDA/ML  (b)  National  Instruments  GRLVQI 

shown  at  the  collected  SNR  shown  at  the  collected  SNR 

Figure  4.14:  National  Instruments  Authorized  ID  results  at  the  collected  SNR 


(a)  National  Instruments  MDA/ML  at  (b)  National  Instruments  GRLVQI  at  the 

the  collected  SNR  collected  SNR 

Figure  4.15:  National  Instruments  Rogue  Rejection  results  at  the  collected  SNR 
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V.  Conclusion 


This  chapter  gives  a  summary  of  the  results  for  Programmable  Logic  Controller 
(PLC)  device  discrimination  using  the  Generalized  Relevance  Learning  from  Vectors 
Quantized  Improved  (GRLVQI)  and  Multiple  Discriminant  Analysis  Maximum  Likelihood 
(MDA/ML)  classifiers  considering  dimensional  reduction  analysis  of  Radio  Frequency 
Distinct  Native  Attribute  (RF-DNA)  time  domain  feature  sets,  using  two  different  receiver 
platforms.  Section  5.1  provides  a  summary  of  the  key  research  activities.  Sections 
5. 1.1 -5. 1.2  provide  a  summary  of  the  research  findings  and  results  comparing  classifier 
performance  based  on  feature  set  dimensionality,  as  well  as  device  discrimination 
performance  based  on  receiver  platform.  Section  5.2  describes  recommendations  for  future 
work  of  PLC  device  hardware  discrimination  and  RF-DNA  fingerprinting. 

5.1  Research  Summary 

Improvement  of  cybersecurity  in  National  Critical  Infrastructure  remains  a  government 
priority.  Supervisory  Control  And  Data  Acquisition  (SCADA)  systems,  which  are  used 
to  control  and  monitor  critical  infrastructure  such  as  waste  water  treatment  centers,  power 
generation  plants,  and  traffic  grids,  are  directly  in  line  with  this  priority.  PLCs  are  a  basic 
unit  of  a  SCADA  system  used  to  control  low-level  operations  such  as  controlling  the  state  of 
a  valve,  monitoring  temperature  or  activating  relays.  As  with  almost  all  electronic  devices 
PLCs  make  use  of  Integrated  Circuits  (IQs  which  can  be  counterfeited  or  manufactured 
with  hardware  trojans  [1,  10].  In  critical  SCADA  applications  potentially  compromised 
hardware  is  a  concern  and  could  inflict  grave  damage.  As  such  PLCs  are  chosen 
to  demonstrate  a  proof  of  concept  demonstration  for  a  hardware  device  discrimination 
method. 

Although  much  work  has  been  done  at  securing  PLCs  at  high  layers  of  the  Open  Systems 
Interconnection  (OSI)  communication  model,  comparatively  less  research  has  been  at  the 
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lowest  layer,  the  physical  waveform  layer.  One  method  of  augmenting  higher  layers  of 
security  by  use  of  the  physical  layer  is  by  exploiting  characteristic  differences  in  waveforms 
inherent  to  a  particular  device  caused  by  component  tolerances  during  manufacturing  .  This 
is  one  focus  of  Air  Force  Institute  of  Technology  (AFIT)s  Radio  Frequency  Intelligence 
(RFINT)  program.  The  program  has  targeted  many  device  and  signal  types  with  the  goal 
of  augmenting  bit-level  security,  enabling  human-like  device  discrimination  and  analyzing 
Side  Channel  Analysis  vulnerabilities  [4,  7,  9,  11-13,  15-20,  23,  25,  27,  38,  39,  43,  46,  49] 

The  goal  of  this  research  was  to  verify  repeatability  of  existing  AFIT  signal  collection 
methods  for  other  receiver  platforms,  as  well  as  to  expand  upon  previous  results 
by  exploring  the  effects  of  fingerprint  feature  dimensional  reduction.  Verification  of 
repeatability  was  accomplished  by  collecting  Unintentional  Radiated  Emission  (URE)  from 
PLC  devices  in  accordance  with  the  collection  procedure  implemented  in  [43]  using  the 
National  Instruments  (NI)  receiver  platform  at  Oak  Ridge  National  Laboratory  (ORNL). 
Dimensional  Reduction  Analysis  (DRA)  was  applied  to  the  NI  signal  collection  as  well  as 
previous  data  sets  collected  using  the  Lecroy  collection  platform  used  at  AFIT.  The  results 
of  these  collections  are  shown  in  Chapter  4. 

Additional  research  contributions  were  made  by  comparing  the  previously  used 
MDA/ML  and  GRLVQI  classifiers  [4,  7,  9,  17,  36,  38,  49]  using  Time  Domain  RF-DNA 
fingerprints,  to  assess  hardware  component  discrimination.  The  classifiers  were  used  to 
perform  classification  of  known  authorized  devices  and  verify  their  claimed  identity,  as 
well  as  detect  and  discriminant  rogue  devices. 

Performance  of  classification  was  assessed  using  an  arbitrary  CAVE= 90%  correct 
classification  baseline  performance  as  consistent  with  previous  AFIT  research  [36]. 
Verification  performance  was  assessed  by  1)  selecting  the  classification  model  with  the 
lowest  Signal  to  Noise  ratio  meeting  the  baseline  performance  2)  generating  Receiver 
Operating  Characteristic  (ROC)  curves  at  the  associated  Signal  to  Noise  Ratio  (SNR)  and 
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evaluating  the  Equal  Error  Rate  (EER)  point  of  True  Verification  Rate  (TVR)>90%  and 
False  Verification  Rate  (FVR)<10%. 

5.1.1  Cross-Platform  Validation 

To  verify  repeatability  of  signal  collection  methods,  PLC  device  emissions  were 
collected  using  an  alternative  receiver,  the  NI  collection  platform.  The  collection  process 
was  successfully  repeated  and  comparable  classification  and  verification  results  were 
obtained.  All  dimensional  fingerprint  sets  considered  in  this  research  met  the  average 
Cave- 90%  correct  classification  baseline  performance  for  both  the  MDA/ML  and  GRLVQI 
classifiers,  albeit  the  sets  achieved  the  baseline  performance  at  varying  SNRs.  Although 
repeatable  results  were  obtained  the  Lecroy  platform  outperformed  the  NI  collection 
platform  results  by  GsA^=10dB  in  the  Full  Dimensional  feature  set  for  benchmark 
performance.  The  Full  Dimensional  baseline  performance  metric  was  used  to  select  the 
SNR  to  evaluate  ROC  curves  for  the  verification  process. 

The  MDA/ML  classifier  performed  best  matched  with  the  Lecroy  receiver  with  a  gain 
of  GSNR^3dB  of  performance  averaged  across  feature  sets  versus  the  GRLVQI  classifier. 
However  the  GRLVQI  classifier  when  used  with  the  NI  collection  platform  saw  a  gain 
of  G5ArR«4dB  of  baseline  performance  averaged  across  feature  sets  versus  the  MDA/ML 
classifier  with  a  gain  of  Gsws=12dB  with  the  Top  10%  feature  set. 

With  the  exception  of  one  feature  set  for  the  Lecroy  platform,  both  receivers  failed  to 
meet  the  EER  for  the  verification  of  all  authorized  device  IDs.  Both  receivers  repeatedly 
failed  to  correctly  verify  device  5  regardless  of  feature  set.  However  both  receivers  achieved 
100%  EER  for  Rogue  Device  Rejection. 

5.1.2  Dimensional  Reduction  Analysis 

Two  types  of  feature  dimensional  reduction  were  considered  in  this  research.  Qualitative 
DRA  feature  sets  were  composed  of  statics  generated  from  only  one  time-domain  signal 
response  {amplitude,  phase,  frequency}.  Features  in  Quantitative  DRA  were  selected  based 
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on  GRLVQI  relevance  rankings.  The  GRLVQI  has  the  inherent  ability  to  rank  features 
used  in  model  development  according  to  their  influence  in  separating  device/class  vector 
representations.  This  provides  an  advantage  over  the  MDA/ML  classifier  as  the  reduction 
of  features  improves  memory  storage,  processing  time,  and  classification  processing 
complexity  with  acceptable  loss  in  classification  performance.  Two  Quantitative  feature 
sets  composed  of  a  percentage  of  the  top  ranked  features  were  considered:  Top  33%  and 
Top  10%.  Both  feature  sets  are  subsets  of  the  Full  dimensional  feature  set,  and  the  Top 
10%  feature  set  is  contained  in  the  Top  33%  feature  set. 

MDA/ML  performance  is  impaired  by  the  reduction  of  feature  dimensionality  for  all 
feature  sets  presented  for  classification  and  verification  in  both  receivers.  The  opposite  is 
true  for  the  GRLVQI  classifier  in  which  the  Top  10  %  feature  set  matched  or  exceeded 
performance  of  the  Full  dimensional  feature  set  based  on  95%  confidence  intervals.  Both 
receiver  platforms  ranked  feature  number  51  as  the  most  influential  feature  for  GRLVQI 
model  development.  Feature  51  is  the  Skewness  of  the  Amplitude  of  Region  9. 

5.2  Future  Work  Recommendations 

The  research  results  presented  here  show  the  effects  of  feature  dimensional  reduction 
in  two  different  receiver  platforms  using  two  different  classifiers.  Both  classification  and 
verificaiton  of  PLC  hardware  device  discrimination  are  shown  to  be  succesful  here  and 
warrant  continued  investigation  including, 

1.  Alternate  RF-Probe:  During  the  signal  collection  process  outlined  in  3.2.2  the 
placement  of  the  RF  probe  requires  precise  alignment  and  any  subsequent  collections 
require  the  probe  re -positioned.  A  less  precise  RF-probe,  more  akin  to  an  antenna 
may  not  require  such  a  rigorous  placement  routine  and  further  mitigate  challenges 
arising  from  repeatability. 


49 


2.  Expansion  of  Feature  Types:  This  research  only  considered  Time-Domain  signal 
response  features.  Previous  AFIT  research  has  shown  other  feature  types  such 
as  Frequency  Domain  features  and  features  derived  from  Gabor  transforms  to  be 
successful  for  RF-DNA  fingerprinting  [4,  36]. 

3.  Alternate  IC  Devices:  Signal  collections  in  this  research  were  taken  from  the 
embedded  microcontroller  on  the  PFC  mainboard.  PFC  device  discrimination  can  be 
further  expanded  by  using  URE  from  other  IC  devices  embedded  on  the  mainboard 
to  develop  RF-DNA  fingerprints. 

4.  Expansion  of  Software  Anomaly  Detection:  Previous  AFIT  research  assessed  PFC 
ladder  logic  operation  verification  using  Correlation  Domain  and  Time  Domain 
features  [43].  PFC  software  anomaly  detection  can  be  further  expanded  by 
considering  the  feature  dimensional  reduction  analysis  demonstrated  in  this  research. 
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VI.  Appendix 


This  appendix  presents  are  the  remaining  device  classification  results  for  both  the 
Lecroy  and  National  Instruments  (NI)  receiver  platforms  for  both  classifiers. 


(a)  Lecroy  MDA/ML  Amplitude  Testing  Results 


(b)  Lecroy  GRLVQI  Amplitude  Testing  Results 


Figure  6.1:  Lecroy  Qualitative  Amplitude  Classification  Results 
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Ave  Pci  Correct  Ave  Pet  Correct 


(a)  Lecroy  MDA/ML  Phase  Testing  Results 


(b)  Lecroy  GRLVQI  Phase  Testing 


Figure  6.2:  Lecroy  Qualitative  Phase  Classification  Results 
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Ave  Pci  Correct  Ave  Pet  Correct 


(a)  Lecroy  MDA/ML  Frequency  Testing  Results 


(b)  Lecroy  GRLVQI  Frequency  Testing  Results 
Figure  6.3:  Lecroy  Qualitative  Frequency  Classification  Results 
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Ave  Pci  Correct  Ave  Pet  Correct 
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(a)  Lecroy  MDAML  Top  33%  Testing  Results 


(b)  Lecroy  GRLVQI  Top  33%  Testing  Results 


Figure  6.4:  Lecroy  Quantitative  Top33  Testing  Averages 


54 
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(a)  Lecroy  MDAML  Top  10  %  Testing  Results 


(b)  Lecroy  GRLVQI  Top  10%  Testing  Results 


Figure  6.5:  Lecroy  Quantitative  Top  10%  Testing  Aves 
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Ave  Pet  Correct  Ave  Pet  Correct 


(a)  NI  MDAML  Amplitude  Testing 


(b)  NI  GRLVQI  Amplitude  Testing 


Figure  6.6:  NI  Amplitude  Testing 


56 


Ave  Pet  Correct  Ave  Pet  Correct 


(a)  NI  MDAML  Phase  Testing 


(b)  NI  GRLVQI  Phase  Testing 


Figure  6.7:  NI  Phase  Testing 
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(a)  NI  MDAML  Frequency  Testing 


(b)  NI  GRLVQI  Frequency  Testing 


Figure  6.8:  NI  Frequency  Testing 
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(a)  NI  MDAML  Top  33%  Testing 


(b)  NI  GRLVQI  Top  33%  Testing 


Figure  6.9:  NI  Top  33%  Testing 
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(a)  NI  MDAML  Top  10%  Testing 


(b)  NI  GRLVQI  Top  10%  Testing 


Figure  6.10:  NI  Top  10%  Testing 
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